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(57) The technique of the present invention extracts 
a plurality of blocks from a master image, processes im- 
age data corresponding to each block by orthogonal 
transform, compares coefficients of adjoining blocks 
with each other, and modifies the order of magnitude of 
the coefficients, based on bit information to be embed- 
ded. The bit information to be embedded is constructed 
as a basic pattern, which is generated, for example, by 
the density pattern method. The basic pattern is embed- 
ded in image data iteratively. The process of iteratively 
embedding the basic pattern enhances the resistance 
against various attacks. The technique of embedding a 
digital watermark in a transformed master image may 
not require labor-consuming management to keep mas- 
ter images closed to the public. 
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Description 

Technical Field 

[0001] The present invention relates to a method of 
embedding a digital watermark in two-dimensional data, 
such as image data, a method of decoding the embed- 
ded digital watermark, and an apparatus that actualizes 
these methods. 

Background Art 

[0002] The advanced computer network like the Inter- 
net accelerates digitalization of information and allows 
a large number of users to readily access required in- 
formation. Such advanced environment, on the other 
hand, causes digital contents including digital informa- 
tion with the copyright to be readily duplicated as unau- 
thorized copies without permission of copyright holders. 
Infringement of the copyright due to the illegal copies is 
becoming a significant problem. With a view to prevent- 
ing the infringement of the copyright with regard to color 
images, which are the main part of digital information in 
the digital contents, the digital watermarking technique 
draws the vigorous attention. The digital watermarking 
technique embeds the copyright information in data of 
a master color image in a secret and unnoticeable man- 
ner to any third person and extracts the copyright infor- 
mation from the embedded master color image accord- 
ing to the requirements. 

[0003] One known technique of embedding a digital 
watermark in two-dimensional data like image data 
causes the image data to be subjected to orthogonal 
transform, such as discrete cosine transform, and 
changes the resulting coefficients based on a predeter- 
mined algorithm. In this technique, the master image is 
closed to the public. The procedure processes both the 
master image and a potential illegal copied image by the 
orthogonal transform and compares their coefficients for 
identification of the copied image. 
[0004] This prior art technique requires the master im- 
age, which is closed to the public, for extraction of the 
copyright information and causes another significant is- 
sue, management of the master image. The increase in 
number of master images makes the management more 
difficult. It is accordingly desirable to embed a digital wa- 
termark in a specific form that allows direct extraction 
from the image. 

[0005] The digital watermark is also required to have 
sufficient resistance against the partial clipping of imag- 
es and overwriting attacks. There are a diversity of at- 
tacks like Laptacian attacks against the digital water- 
marking that utilizes the orthogonal transform. No prac- 
tically effective techniques have been proposed so far 
to solve the problems like the attacks and the partial clip- 
ping of images. 


Disclosure of the Invention 

[0006] The object of the present invention is to solve 
the problems arising in the prior art technique of embed- 

5 ding and decoding the digital watermark utilizing the or- 
thogonal transform discussed above, and thus to pro- 
vide a technique of embedding and decoding the digital 
watermark that ensures good preservation and high re- 
sistance against a diversity of attacks. 

w [0007] At least part of the above and the other related 
objects is attained by a method of embedding a digital 
watermark in a master image. The method includes the 
steps of: extracting blocks of a predetermined size from 
the master image; processing image data correspond- 

'5 ing to each block by orthogonal transform; comparing 
orthogonal transformed coefficients of at least two 
blocks having a predetermined relationship with each 
other and making the coefficients satisfy a preset order 
of magnitude according to bit information specified as 

20 the digital watermark, so as to embed the bit information; 
and processing each block with the embedded bit infor- 
mation by inverse orthogonal transform, so as to output 
a resulting image with the digital watermark embedded 
therein. 

25 [0008] The method of embedding the digital water- 
mark according to the present invention uses the at least 
two blocks having a predetermined relationship. The 
method compares the coefficients of the at least two 
blocks obtained by orthogonal transform and embeds 
30 bit information according to a preset order of magnitude 
of the coefficients . Only the person who knows both the 
predetermined relationship between the at least two 
blocks and the preset order of magnitude of the coeffi- 
cients can extract the digital watermark from the image. 
35 Any third person who does not know the preset order of 
magnitude of the coefficients, on the other hand, can not 
decode the digital watermark. This arrangement thus 
actualizes the method of embedding the digital water- 
mark, which is easily processed and has high resistance 
40 against various attacks. One possible application keeps 
a master image with the digital watermark embedded 
therein closed to the public and identifies the digital wa- 
termark by comparison with the master image. 
[0009] The present invention is also directed to a 
45 method of decoding a digital watermark from a master 
image with the digital watermark embedded therein . The 
decoding method includes the steps of: extracting 
blocks of a predetermined size from the master image; 
processing image data corresponding to each block by 
so orthogonal transform; and comparing orthogonal trans- 
formed coefficients of at least two blocks having a pre- 
determined relationship with each other and extracting 
bit information, based on a preset order of magnitude 
that is applied to the coefficients. 
55 [001 0] The method of decoding the digital watermark 
according to the present invention enables the digital 
watermark to be readily extracted from the image. 
[001 1] Based on the above principle, the technique of 
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the present invention may be actualized by a diversity 
of embodiments. For example, the at least two blocks 
having the predetermined relationship are blocks ad- 
joining to each other. Adjoining images often hold a pre- 
determined relationship. This may give a specific ten- 
dency to the order of magnitude of the coefficients with 
regard to the adjoining blocks. The adjoining blocks are 
thus suitable for the method of embedding the digital wa- 
termark according to the present invention. 
[0012] Discrete cosine transform may be used for the 
orthogonal transform of the image data. The discrete co- 
sine transform is adopted in JPEG and is favorably used 
for image compression. One preferable procedure 
quantizes the coefficients obtained by the orthogonal 
transform with a quantization table and uses the quan- 
tized coefficients to embed the bit information. The com- 
pression efficiency is freely adjustable by modifying the 
quantization table. 

[0013] In accordance with one preferable embodi- 
ment, the image data is converted into a system of a 
luminance Y and color differences Cb and Cr, prior to 
the orthogonal transform. The luminance Y and the color 
differences Cb and Cr are subjected to the discrete co- 
sine transform as the orthogonal transform. The bit in- 
formation is embedded in coefficients obtained by the 
discrete cosine transform of the luminance Y. 
[001 4] I n accordance with another preferable embod- 
iment, embedding the bit information is carried out when 
the quantized coefficients of the at least two blocks are 
not all equal to zero. In the case where the coefficients 
of these blocks are all equal to zero, embedding the bit 
information to change the coefficients to a value other 
than zero may lower the efficiency of data compression. 
[0015] In accordance with one preferable application, 
the method of embedding the digital watermark further 
includes the step of introducing a logic function that is 
true when a difference between the orthogonal trans- 
formed coefficients of the at least two blocks having the 
predetermined relationship is in a preset range. This ad- 
vantageously simplifies the operation. In this case, the 
procedure adopted to embed the bit information is mod- 
ified, based on the true and false state of the logic func- 
tion. This facilitates change of the preset range. 
[0016] In the above application, the method may fur- 
ther include the steps of: providing a secret key corre- 
sponding to each coefficient obtained by the orthogonal 
transform; and modifying the procedure adopted to em- 
bed the bit information, based on the secret key corre- 
sponding to each coefficient and the true and false state 
of the logic function with regard to the coefficient. This 
application makes it practically impossible to specify the 
rule adopted to embed the bit information without the 
secret key. The secret keys may be under management 
corresponding to respective images or respective cop- 
yright holders. Such arrangement facilitates manage- 
ment of the digital watermarking. 
[0017] In accordance with another preferable applica- 
tion, the embedding method further includes the steps 


of: 

providing a basic pattern, which is defined in a 
two-dimensional manner as a combination of binary in- 
formation, as information of the digital watermark; spec- 
5 ifying each piece of binary information included in the 
provided basic pattern as the bit information to be em- 
bedded; and embedding the binary information of the 
basic pattern by setting the at least two blocks having 
the predetermined relationship to one unit. In this appli- 
10 cation, the digital watermark is embedded as the basic 
pattern defined in the two-dimensional manner, and this 
basic pattern may directly be used as the copyright in- 
formation. It is preferable that the basic pattern has re- 
dundancy. The redundancy of the pattern is utilized for 
'5 detecting errors. The bit information of the basic pattern 
used for digital watermarking may be varied according 
to the characteristics of a master image, in which the 
digital watermark is to be embedded. A typical example 
of the basic pattern having redundancy is a density pat- 
tern. The density pattern is based on the principle that 
there are a plurality of different dot layouts having an 
identical density in the range of nxn pixels, and has high 
redundancy. The high redundancy of the density pattern 
is effectively used to detect errors. 
[001 8] When the number of elements constituting the 
basic pattern is sufficiently greater than the number of 
extracted blocks, the basic pattern may be embedded 
in the image data iterativety a predetermined number of 
times. This arrangement allows the basic pattern to be 
accurately decoded even when part of the image is 
clipped, thus enhancing the resistance of digital water- 
marking. 

[0019] The method of embedding the digital water- 
mark and the method of decoding the digital watermark 
may be attained in the form of storage media, in which 
programs of the corresponding methods are stored. 
Typical examples of the storage media include CD- 
ROMs, memory cards, and flexible disks. Such pro- 
grams may be downloaded via a communication tine. 
The method of embedding the digital watermark may be 
incorporated in a retouching software program for 
processing image data integrally or as a plug-in software 
program. The embedding method and the decoding 
method may alternatively be actualized by independent 
programs for embedding the digital watermark and de- 
coding the digital, watermark. The color images deliv- 
ered as the digital contents often undergo compression 
by a diversity of compression techniques like JPEG. The 
programs of the corresponding methods may thus be 
incorporated in image data compression software inte- 
grally or as plug-in software programs. The programs 
may integrally be incorporated in a scanner that reads 
printed image data as digital data or scanning software. 
The programs may also be incorporated in a printer driv- 
er and a printer used to print digital data. The embedding 
method and the decoding method may also be actual- 
ized as apparatuses, in this case, these methods may 
be implemented by exclusive apparatuses, or the com- 
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puter may read programs from the storage media to ac- 
tualize a digital watermark embedding apparatus or a 
digital watermark decoding apparatus. 

Brief Description of the Drawings 

[0020] 

Fig. 1 illustrates the principle of a procedure of em- 
bedding and decoding a digital watermark in an em- 
bodiment of the present invention; 
Fig. 2 is a block diagram illustrating the structure of 
a digital watermark embedding apparatus 10 of the 
embodiment; 

Fig. 3 is a flowchart showing a watermark pattern 
generation routine; 

Fig. 4 shows a watermark pattern to be embedded; 
Fig. 5 shows various pieces of information express- 
ible by the watermark pattern; 
Fig. 6 is a flowchart showing the outline of a routine 
of embedding a digital watermark; 
Fig. 7 is a flowchart showing a routine of embedding 
bit information; 

Fig. 8 is a table used to identify output logic func- 
tions E1 and E2, based on bit information s to be 
embedded, a logic variable D, and key information 
K; 

Fig. 9 is a flowchart showing a routine of decoding 
a digital watermark; 

Fig. 1 0 is a flowchart showing a routine of decoding 
embedded bits; 

Fig. 11 is a table used to identify an output logic 
function EE for decoding; 

Fig. 12 is a flowchart showing a routine of decoding 
copyright management information; 
Fig. 13 illustrates a basic pattern used in an exper- 
iment; 

Fig. 14 illustrates a watermark pattern obtained by 
iteratively laying out the basic pattern in the exper- 
iment; 

Fig. 15 shows an image with the watermark pattern 

embedded therein in the experiment; 

Fig. 16 shows image data after a Laplacian attack; 

and 

Fig. 17 illustrates the watermark pattern decoded 
from the image data exposed to the Laplacian at- 
tack. 

Best Modes of Carrying Out the Invention 

<Principle of Embedding and Decoding Digital 
Watermark> 

[0021] Fig. 1 illustrates a procedure of embedding a 
digital watermark in a digital image in an embodiment of 
the present invention and the principle of such process- 
ing. Image data of a master image are stored as digital 
information. As illustrated, the procedure extracts imag- 


inary blocks of 8x8 pixels from the master image. This 
is the technique widely used for image compression like 
JPEG. Although the size of each block is 8x8 pixels in 
this embodiment, any other settings may be applicable 

5 for the size of the block. 

[0022] The procedure subsequently causes the mas- 
ter image data to be subjected to color conversion. The 
process of color conversion may be omitted if not re- 
quired. The color conversion, for example, converts 

10 RGB image data typically used on the computer into a 
YCrCb color difference signal system for the conven- 
ience of subsequent processing. The color conversion 
is carried out only for color images. When the master 
image is monochromatic, only luminance signals are the 

15 object of the processing, so that no color conversion is 
required. The subsequent processing may be per- 
formed without color conversion of the master image, 
when the efficiency of data processing is not much of 
importance. 

20 [0023] In this embodiment, the color image is convert- 
ed into the YCrCb signal system. The procedure carries 
out orthogonal transform of the Y plane representing a 
luminance signal. The procedure of this embodiment 
applies discrete cosine transform (hereinafter referred 
25 to as DCT) for the orthogonal transform, though another 
transform technique is also applicable. The DCT is car- 
ried out for each imaginary block of 8x8 pixels. Division 
of DCT coefficients (8x8) obtained by the DCT by co- 
efficients provided in a quantization table (quantization 
30 levels) gives quantized DCT coefficients (8x8). The di- 
vision based on the quantization table aims to compress 
data and is not essential, in principle, for embedding the 
digital watermark. The quantization table depends upon 
the roughness of the quantization. It is, however, not de- 
35 sirable that embedding the digital watermark significant- 
ly deteriorates the picture quality. A table giving little de- 
terioration of the picture quality (that is, a table of a low 
compression ratio) may thus be applied favorably for the 
quantization table. 
40 [0024] The above procedure carries out the DCT for 
each block of 8x8 pixels and obtains the quantized DCT 
coefficients. The procedure subsequently performs 
comparison between the DCT coefficients with regard 
to adjoining two blocks, so as to embed bit information. 
45 Because of the simplicity of the processing and the con- 
tinuity in the master image, adjoining blocks are gener- 
ally selected for the object of the comparison of the DCT 
coefficient. The object of the comparison is, however, 
not restricted to the adjoining blocks. The two blocks 
so may be adjacent to each other in a vertical direction or 
in a horizontal direction. Another modification may uti- 
lize the DCT coefficients of three or more blocks to em- 
bed the bit information. The bit information embedded 
here is obtained from a basic pattern created in ad- 
55 vance. The details of the method of creating the basic 
pattern, which is the source of the bit information, and 
the method of embedding the bit information will be dis- 
cussed later. 
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[0025] The procedure then causes each block with 
the bit information embedded therein to be subjected to 
inverse discrete cosine transform (hereinafter referred 
to as IDCT), The IDCT uses the same quantization table 
as that used for the DCT The inverse transform recon- 
verts each block into image data of the original YCrCb 
color difference signal system. Application of the quan- 
tization table generally loses part of a high frequency 
component of the image data and thereby deteriorates 
the picture quality. The procedure subsequently carries 
out inverse color conversion to give a processed image 
of the original RGB system. This series of processing 
gives a processed image with the basic pattern embed- 
ded therein as the digital watermark. 
[0026] In the above description, the processed image 
is reconverted to the original format (the RGB color sys- 
tem). The processed image may not be reconverted to 
the original format but may be stored in another format 
like JPEG. In this case, the procedure may embed a ba- 
sic pattern, which corresponds to specific watermark in- 
formation, in the image stored in the format of JPEG. 
[0027] The procedure of decoding the watermark in- 
formation embedded in the image data follows the 
above series of processing in the reverse direction. The 
procedure first specifies quantized DCT coefficients 
with regard to two blocks in the image data, identifies 
the relationship between the quantized DCT coefficients 
of the two blocks, and thereby extracts the bit informa- 
tion embedded in the two blocks. The procedure then 
rearranges the extracted bit information to restore the 
basic pattern. The details of this procedure will be dis- 
cussed later. 

<General Structure of Apparatus> 

[0028] The structure of an apparatus used in the em- 
bodiment is described briefly. Fig. 2 is a block diagram 
illustrating the hardware structure of a digital watermark 
embedding apparatus 10 that implements the method 
of embedding the digital watermark in the embodiment. 
As illustrated, the digital watermark embedding appara- 
tus 1 0 includes a computer 20 and a scanner 12, an ex- 
ternal storage device 14, such as CD-ROMs, a modem 
16, and a display monitor 18 that are all connected to 
the computer 20. The computer 20 includes a CPU 22 
that executes processing, a ROM 23 that stores a mon- 
itor program and other programs therein, and a RAM 24 
onto which programs are loaded or in which data are 
temporarily registered in the course of operations. The 
computer 20 also includes a frame memory 25 that func- 
tions to display images on the monitor 18, a controller 
(HDC) 26 that is in charge of data transmission to and 
from the external storage device 1 4, a serial input-output 
circuit (SIO) 28 that is in charge of data transmission to 
and from the modem 1 6, and an input interface (input 
IF) 29 that functions to input images read by the scanner 
12. Akeyboard and a mouse are also connected to the 
computer 20, although they are omitted from the illus- 


8 

tration. The modem 1 6 is connected to a computer net- 
work via a communication line NT The computer 20 may 
download various image processing programs from a 
non-illustrated server on the computer network via the 
5 modem 1 6. The computer 20 may distribute image data 
with a digital watermark embedded therein through the 
computer network. 

[0029] The CPU 22 executes the programs loaded 
onto the RAM 24 to actualize the functions of a block 
10 extraction unit 31 , a transformation unit 32, a bit infor- 
mation embedding unit 33, and an output unit 34. These 
unit respectively correspond to the block extraction 
means, the transformation means, the bit information 
embedding means, and the output means of the present 
'5 invention. The detailed functions of these units will be 
discussed later as a series of processing executed by 
the CPU 22. 

[0030] The following description regards the method 
of embedding the digital watermark. Before the digital 
watermark is embedded in the image data, the process 
generates a watermark pattern to be embedded. Fig. 3 
is a flowchart showing a watermark pattern generation 
routine. When the program enters this routine, the CPU 
first inputs copyright information and extends the input 
copyright information to decimal digits (step S50). The 
copyright information is used to identify the copyright 
holder of a master image, which is the object of digital 
watermarking. In this embodiment, information corre- 
sponding to one code is assigned to each copyright 
holder. One code may alternatively be assigned to each 
work. 

[0031] The process of extending decimal digits intrin- 
sic to each copyright holder may automatically be car- 
ried out in the computer 20. Alternatively the copyright 
holder may manually specify a desired code. One pref- 
erable procedure transmits required information to a to- 
tal management center, for example, a copyright man- 
agement center, via the modem 1 6 in response to input 
of a copyright holder name into the computer 20 and 
receives the code intrinsic to the copyright holder or a 
work of the copyright holder from the total management 
center. This allows total management of copyright hold- 
er information. 

[0032] The CPU then generates a basic pattern, 
based on the decimal digits (step S60). The technique 
of this embodiment does not simply embed a digital wa- 
termark as bit information but embeds the digital water- 
mark in a specific pattern. The concrete procedure of 
this embodiment adopts the density pattern method 
(more specifically the method of Nakamura) to generate 
the basic pattern from the decimal digits. In the case 
where the watermark information is embedded in an im- 
age as a series of letter or numeral bits, no redundancy 
of the watermark information results in decoding errors 
when the image with the embedded watermark informa- 
tion is exposed to an attack. One possible measure 
against such troubles applies an error correction code 
to the digital watermark. The technique of this embodi- 
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ment, however, utilizes the redundancy of the pattern. 
[0033] The basic pattern is generated by the following 
method. The description first regards how much infor- 
mation can be mounted on a binary figure having a pre- 
determined size n (in this example, 4x4, that is, n=4 as 
shown in Fig. 4). In the example of 4x4 shown in Fig. 
4, the basic pattern consists of 1 6 watermark signals s 

(u) (u= 0, 1 15). The signals s(u) are binary data 

taking either the value '0' or the value T. The number 
of s(u)= 1 is expressed by m, which is introduced to map 
a binary pattern to the data using a redundancy code 
, m-out-of-n2'. The value of each element s(u) is deter- 
mined recursively by regarding each of the n2 elements 
on the watermark signal S to be embedded as a binary 
digit weighted by the number of combinations and using 
the relationship of: 

n2Ilm = n2-inm-1 +n2-inm (1) 

Here the expression 'anb' represents the number of 
combinations in the case of extracting b elements out of 
a elements (the same definition is adopted in the follow- 
ing explanation). 

[0034] When u= 0, the first term on the right side of 
Equation (1) represents the total number of arrays sat- 
isfying s(0) <r- 1 , and the second term represents the 
total number of arrays satisfying s(0) <- 0. The arrays 
satisfying s(0)= 1 are accordingly mapped to integers of: 

{0,1,2, m2-inm-1}, and 
the arrays satisfying s(0)= 0 are mapped to integers of 

{n2-inm-1, n2-inm-1 + 1, ..... m2nm - 1}. 
According to this rule, the result of the comparison be- 
tween copyright management information t (decimal 
digits) and n2-1Tlm-1 is specified as: 

when t < n2-inm-1, s(u) <- 1; and 

when t > n2-1 nm-1 , s(u) <- 0. 
[0035] If s(0)= 1 , the procedure decrements the value 
m by one to lay out the residual (m-1 ) signals having the 
value 'V out of the signals s(u) other than s(0) (u= 1, 
2, ...,n2-1), and then compares t with n2-2nm-1. lfs(0) 
= 0, on the contrary, the procedure subtracts n2-1Ilm-1 
from the value t to lay out the m signals having the value 
T out of the signals s(u) other than s(0) (u= 1, 2, ... p 
n2-1 , and then compares t with n2-2nm-1 . This opera- 
tion is repeated until m= 0. This series of processing de- 
termines all the positions of the value '1' in the basic 
pattern. The value '0* is accordingly set to the remaining 
s(u). This gives a watermark pattern S, which is one-to- 
one mapped to the given values m and t. 
[0036] Fig. 5 shows a variety of basic patterns S ex- 
pressible by the above procedure. When m is in a range 
of 7 to 9, one basic pattern is capable of expressing in- 
tegers t in decimal notation corresponding to 13 bits. In 
the case where the basic pattern has high redundancy 
like the density pattern used in this embodiment, the 
high redundancy may be utilized to implement another 
series of processing, such as detection of errors. There 


are a plurality of density patterns having an identical 
density. In the density pattern having a fixed density, the 
setting for the layout of on (black) bits has a high degree 
of freedom. Under the condition of a fixed density, one 

5 possible application may restrict the number of on 
(black) pixels among all the pixels in each raster line 
(that is, the total number of pixels in the direction x) to 
even numbers. Another application may select an ap- 
propriate density pattern by taking into account the ef- 

10 fects on the picture quality. The procedure of this em- 
bodiment uses the density pattern having the specific 
size to allow expression of decimal digits. Only one ba- 
sic pattern is sufficient when the value t is in the express- 
ible range of the redundancy system *m-out-of-n2'. In the 

'5 case where the copyright information, which is the ob- 
ject of management, exceeds this expressible range, 
the basic pattern may be modified to two or three differ- 
ent layouts, prior to embedding. 
[0037] In this embodiment, the density pattern is ca- 

20 pable of expressing decimal digits, so that the process- 
ing of step S60 shown in Fig. 3 selects and generates 
one of the basic patterns corresponding to the decimal 
digits obtained at step S50. The CPU repeats the se- 
lected basic pattern over the whole range of the image 

25 to generate the watermark pattern (step S70). The 
processing of this step is carried out as discussed below. 
[0038] It is here assumed that a master image P, in 
which the watermark pattern is to be embedded, con- 
sists of 256 pixels x 256 pixels. There are 32x32 blocks 

30 Bu having the size of 8x8 pixels in the master image P. 
When each pair of adjoining blocks Bu and Bu+1 among 
all the blocks is specified as one set, there are 32X16 
sets. Each bit of s(u) is mapped to each set, and the 
master image P accordingly has 8x4 spaces, each ac- 

35 cepting the 4x4 basic pattern. When only one basic pat- 
tern is sufficient for expression of copyright manage- 
ment information, the watermark information S can be 
recorded iteratively by the total of 32 times, that is, four 
times in the horizontal direction and eight times in the 

to vertical direction, in the master image P. The arrange- 
ment of recording the basic pattern iteratively to gener- 
ate the watermark pattern enhances the redundancy of 
the watermark pattern and attains the sufficiently high 
resistance against Laplacian attacks on the image data 

45 with the digital watermark embedded therein and partial 
clipping of the image data. In the case where two or 
more basic patterns S are required to express a large 
capacity of copyright management information, the two 
or more basic patterns S are generated by varying the 

so value m to be distinguishable from each other. Since 
there is a possibility that part of the image is clipped, it 
is preferable that the plurality of basic patterns S are laid 
out at random in the 8x4 spaces. This application ena- 
bles not only the decimal digits but a letter or a series 

55 of letters to be embedded. In the case of a large image, 
the procedure may increase the number of repetitions 
of the basic pattern and embeds detailed letter informa- 
tion representing the copyright management informa- 


1 

tion. 

[0039] The above series of processing generates the 
watermark pattern. The following describes the details 
of the process of embedding the watermark pattern in 
image data. Fig. 6 is a flowchart showing the outline of 
a routine of embedding the digital watermark. When the 
program enters the routine, the CPU first reads master 
image data (step S1 00), and then carries out color con- 
version of the input master image data (step S110). As 
described previously, the color conversion converts the 
RGB signal system, which is typically used for the image 
processing on the computer 20, into the YCrCb color 
difference signal system. When the master image is ex- 
pressed by color signals, the color conversion gives im- 
age data divided into three planes, that is, a luminance 
signal Y and color difference signals Cr and Cb. 
[0040] The CPU subsequently extracts blocks of 8x8 
pixels from the Y plane corresponding to the luminance 
signal (step S120). When the master image consists of 
256x256 pixels, for example, the procedure sequential- 
ly extracts 32 blocks in the horizontal direction x from 
the upper left corner of the master image as the origin, 
shifts in the vertical direction y when reaching the end 
of the horizontal line, and again extracts 32 blocks on a 
next horizontal line. This series of operations is repeated 
32 times to extract 32x32 blocks from the master im- 
age. Each extracted block in the Y plane is subjected to 
discrete cosine transform (DCT). The details of the DCT 
are known in the art and are thus not specifically de- 
scribed here. The procedure of this embodiment uses a 
quantization table for the luminance component of 
JPEG to quantize the DCT coefficients. 
[0041] The CPU then embeds the bit information 
specified according to the preset watermark basic pat- 
tern in the DCT coefficients of each block (step S150). 
The details of this process will be discussed later with 
reference to Fig. 7. After embedding the bit information, 
the CPU carries out inverse discrete cosine transform 
(IDCT) of the 32x32 blocks ( step S170), performs re- 
verse color conversion to reconvert the image data into 
the original RGB system (step S180), and outputs the 
resulting image (step S190). The output may print the 
image with a color printer or distribute the image to the 
users on the network via the modem 16. Although the 
procedure of this embodiment reconverts the image with 
the embedded watermark pattern into the RGB system, 
the compressed image in the JPEG format may alterna- 
tively be output. 

[0042] The details of the bit information embedding 
process (step S150) are discussed below. When the 
program enters the bit information embedding routine 
shown in Fig. 7, the CPU first inputs DCT coefficients Di 
(u,v) and Di+1(u,v) with regard to u-th and (u+1)-th 
blocks adjoining to each other in the Y plane (step 
S151). The CPU then divides and quantizes the input 
DCT coefficients with a quantization table, specifies a 
frequency coefficient domain F, and identifies elements 
fu(i j) and fu+1 (i,j) of the frequency coefficient domain F 


12 

(step S152). Here i, j= 0, 1, 7. The CPU then calcu- 
lates an absolute value d(i,j) of the difference between 
the coefficients at the identical address (i j) (step S1 53). 
[0043] The CPU subsequently specifies key informa- 

5 tion K (step S 154). The key information K is used to keep 
the embedding rule closed to the public in the process 
of embedding the bit information discussed below, and 
is preset in secrecy as K = {k(i,j)lij= 0, 1 , 2, .... 7}. In this 
embodiment k(i,j) takes either the value *0' or the value 

w t. 

[0044] The CPU extracts bit information s(u), which is 
to be embedded, from the watermark pattern S (step 
S155). In this embodiment, one piece of bit information 
is mapped to each pair of adjoining blocks. The setting 
'5 based on the watermark pattern discussed previously 
(see Fig. 3) maps each piece of bit information to each 
set of adjoining blocks. The CPU then calculates a logic 
variable D (step S156). The setting of the logic variable 
D is specified according to the following condition. When 
d(ij) is not less than a value 'a', D is set equal to 1 . When 
d(i,j) is less than the value 'a', on the other hand, D is 
set equal to 0. The value 'a' represents the resistance 
of digital watermark. The greater value 'a* generally 
gives the higher resistance of the digital watermark 
against an external attack. The extremely large value 
'a', however, adversely affects the picture quality. 
[0045] The above series of processing identifies the 
logic variable D based on the absolute value of the dif- 
ference between the quantized DCT coefficients, the 
key information K, and the bit information s(u) to be em- 
bedded, with regard to a set of adjoining blocks. The 
CPU subsequently calculates an output logic function 
E1 from these three pieces of information (step S161). 
The output logic function E1 is shown in a logic table of 
Fig. 8 and is defined by a logic expression of: 

E1 =SVDVK\+S-DVK 

The symbol V shows that a preceding variable is nega- 
tive logic. 

[0046] The CPU determines whether or not the output 
logic function E1 is equal to 1 (step S162). When E1 is 
not equal to 1, the next step is skipped. When E1 is equal 
to 1 , on the other hand, the CPU adds a preset value e 
(i,j) to the greater one of the coefficients fu(i j) and fu+1 
(i,j) at step S1 63. The preset value e(i,j) is generally set 
equal to the value 'a' representing the resistance of the 
digital watermark. 

[0047] The CPU subsequently calculates an output 
logic function E2 (step S1 65). This process is equivalent 
to identify the box E2 shown in Fig. 8, and the output 
logic function E2 is defined by a logic expression of: 

E2 = SVD-K + S-D-KV 
The CPU then determines whether or not the output log- 
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ic function E2 is equal to 1 (step S1 66), as in the case 
of the output logic function E1 . When E2 is equal to 1, 
the CPU subtracts the preset value e(i,j) from the greater 
one of the coefficients fu(ij) and fu+1(i,j) at step S167. 
The CPU determines whether the above series of 
processing has been performed for all the blocks includ- 
ed in the master image (step S169), and repeats the 
above series of processing until the processing has 
been concluded for all the blocks. In the case of the mas- 
ter image consisting of 256x256 pixels, since each ob- 
ject of the processing is the set of two adjoining blocks, 
the above series of processing is repeated 32x16 
times. In the course of the processing, the 4x4 basic 
pattern is thus iteratively embedded 4x8 times, as men- 
tioned previously. 

[0048] As described above with reference to the prin- 
ciple diagram of Fig. 1 , the outline of the watermark pat- 
tern embedding process shown in Fig. 6, and the details 
of the bit information embedding process shown in Fig. 
7, the technique of this embodiment utilizes the quan- 
tized DCT (discrete cosine transform) coefficients of the 
two blocks having a predetermined relationship (the 
contiguity in the x direction in this embodiment). When 
there is a certain difference between these coefficients, 
the procedure embeds the bit information s(u) in the co- 
efficients with referring to the key information K. Namely 
this method regards the one-bit watermark signal s(u) 
as the value of the difference between the pair of ele- 
ments f u(ij) and f u+1 (i,j) in the frequency coefficient do- 
main and embeds the watermark signal s(u) with the re- 
dundancy in the quantized coefficients with regard to 
each set of the two blocks. The watermark pattern is 
constructed as repetition of the basic pattern. This at- 
tains the dual structure of the watermark information and 
allows introduction of rich redundancy. The digital wa- 
termark accordingly has sufficiently high resistance 
against clipping and Laplacian attacks. The setting of 
the resistance may be specified readily by the preset 
value e(ij). The resistance against an attack is dis- 
cussed later as an experiment. Since the pair of ele- 
ments fu(ij) and f u+1 (i,j) in the frequency coefficient do- 
main are varied, the image is affected by a minute var- 
iation in frequency component over the whole frequency 
range and slightly deteriorates its picture quality. When 
a high-quality output image is required, a small value 
should be set to the resistance parameter e(i,j)= a. 
Omission of the coefficients having i and j equal to 0, 
that is, the direct current component, and part of low fre- 
quency components from the target of embedding the 
bit information is also preferable to keep the high picture 
quality. In this application, while several elements hav- 
ing i j= 0 are omitted from the target of embedding the 
bit information, the bit information is still embedded in 
most of the blocks. The digital watermark can thus be 
restored with sufficient accuracy. 
[0049] Although using the key information K is not es- 
sential, the procedure of this embodiment uses the key 
information K to make the embedding rule closed to the 


public. This makes it practically impossible for any third 
person who gains the image to illegally extract and de- 
stroy the embedded information. 
[0050] The following describes a process of decoding 

5 a digital watermark embedded in an image. Fig. 9 is a 
flowchart showing a routine of decoding a digital water- 
mark. Fig. 10 is a flowchart showing the details of the 
process of decoding embedded bits. These flowcharts 
correspond to the flowcharts of the digital watermark 

10 embedding process shown in Figs. 6 and 7 and are ex- 
plained briefly. 

[0051] When the program enters the digital water- 
mark decoding routine, the CPU first reads image data, 
which is the object of decoding the digital watermark 

*5 (stepS200), and carries out color conversion of the input 
image data (step S21 0). The CPU then extracts blocks 
of 8x8 pixels from the Y plane (step S220) and proc- 
esses the extracted blocks by discrete cosine transform 
(DCT) (step S230). The CPU decodes embedded bits 

20 from the quantized DCT coefficients thus obtained (step 
S250) and restores a basic pattern of the digital water- 
mark based on the decoded bit information (step S270). 
The CPU finally outputs a restored digital watermark 
(step S290). 

25 [0052] As shown in Fig. 10, the process of decoding 
the embedded bits at step S250 first inputs DCT coeffi- 
cients D'i (u,v) and D'i+1 (U,v) of adjoining blocks in the 
Y plane (step S251). The CPU divides and quantizes 
the DCT coefficients with a quantization table, specifies 

30 a frequency coefficient domain F\ and identifies ele- 
ments f'u(i,j) and f'u+1(ij) of the frequency coefficient 
domain F' (step S252). The CPU subsequently calcu- 
lates an absolute value d'(ij) of the difference between 
the coefficients at the identical address (i j) (step S253). 

35 [0053] The CPU then specifies the key information K 
(step S254) and calculates a logic variable D from the 
absolute value d'(i.j) (step S256). The setting of the logic 
variable D is specified according to the following condi- 
tion. When d'(ij) is not less than the value 'a', D is set 

40 equal to 1 . When d'(i j) is less than the value 'a', D is set 
equal to 0. 

[0054] The above series of processing identifies the 
logic variable D based on the absolute value of the dif- 
ference between the quantized DCT coefficients and the 
45 key information K with regard to a set of two blocks. The 
CPU then calculates an output logic function EE from 
these two pieces of information (step S261). The output 
logic function EE is shown in a logic table of Fig. 11 and 
is defined by a logic expression of: 

50 

EE = D-K + DVK\ 

[0055] The CPU determines whether or not the output 
55 logic function EE is equal to 1 (step S262). When EE is 
equal to 1 , a variable g is incremented by one (step 
S263). When EE is not equal to 1 , on the other hand, a 
variable z is incremented by one (step S264). The CPU 
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determines whether the above series of processing has 
been performed for all the blocks included in the master 
image (step S266), and repeats the above series of 
processing until the processing has been concluded for 
all the blocks. 

[0056] On completion of the above processing with re- 
gard to all the blocks, the CPU compares the variable g 
with the variable z and sets either the value T or the 
value '0* to the bit information s(u) embedded in the two 
blocks according to the majority rule (steps S268 and 
S269). 

[0057] The above series of processing enables the bit 
information s(u) embedded in the adjoining two blocks 
to be extracted easily and stably. When all the pieces of 
bit information s(u) corresponding to u= 0, 1, 2, .... 15 
are identified to specify the basic pattern of the digital 
watermark, the process decodes the copyright manage- 
ment information. Fig. 12 is a flowchart showing a rou- 
tine of decoding copyright management information. 
When the program enters this routine, the CPU first uses 
variables x and y, which may take values in a range of 
1 to 16, and substitutes a number of combinations xny 
of the variables x and y into an array M(x,y) (step S271 ). 
The CPU then initializes the values x, y, u, and t (step 
S272), prior to a series of processing discussed below. 
[0058] At step S273, it is determined whether or not 
the variable y, which is initialized to the value m, is equal 
to 0. This determines conclusion of the retrieval. Imme- 
diately after a start of this routine, the value of the vari- 
able y is not equal to 0. The program accordingly pro- 
ceeds to step S274, at which it is determined whether 
or not the bit information s(u) is equal to 0. When the bit 
information s(u) is equal to 0, an array M(x,y-1 ) is added 
to the decimal digits corresponding to the copyright 
management information (step S275). When the bit in- 
formation s(u) is not equal to 0, on the other hand, the 
variable y is decremented by one (step S276). In either 
case, the variable x is decremented by one while the 
variable u is incremented by one (step S278). 
[0059] The CPU subsequently determines whether or 
not the variable u exceeds the value '15' (step S279). 
When the variable u exceeds 1 5 or when the variable y . 
is equal to 0, the program shifts to step S280 to decode 
the lower bits of the management inform ationt as log 2 M 
(16J). The above series of processing decodes the nu- 
merical data, that is, the copyright management infor- 
mation t, embedded in the basic pattern of the digital 
watermark. The technique of this embodiment specifies 
the basic pattern and repeats the basic pattern a plural- 
ity of times to embed the digital watermark. This dual 
structure ensures introduction of rich redundancy and 
enables the digital watermark to be decoded easily and 
stably. 

[0060] An experiment was made to embed the copy- 
right management information according to the proce- 
dure of the embodiment discussed above. In this exper- 
iment, t=1 23 (decimal digits) was used as the copyright 
management information T. The value m=3 was adopt- 


ed to express the value t by taking into account the result 
shown in Fig. 5. Fig. 13 shows the basic pattern gener- 
ated by the method of the embodiment. The generated 
basic pattern was laid out iteratively in the area of 

5 256x256 at the ratio of one to every two blocks as 
shown in Fig. 3. This gave a watermark pattern shown 
in Fig. 14. This binary image was embedded as the wa- 
termark information in the DCT frequency coefficient do- 
main with regard to a variable-density image P. The em- 

10 bedded result is shown in Fig. 1 5. Fig. 1 6 shows the re- 
sult of a Laplacian attack against the resulting image P' 
with the watermark information embedded therein. The 
watermark information was decoded from the image ex- 
posed to the Laplacian attack. The decoded result is 

*s shown in Fig. 17. As clearly understood from the illus- 
tration, the original watermark pattern is mostly pre- 
served through the Laplacian attack. Although part of 
the watermark pattern was lost, it was easy to accurately 
restore the original basic pattern from this attacked iter- 

20 ative pattern by utilizing, for example, the principle of 
majority rule. 

[0061] The embodiment of the present invention and 
the results of the experiment are described above. The 
computer programs that attain the functions of the block 

25 extraction unit 31 , the transformation unit 32, the bit in- 
formation embedding unit 33, and the output unit 34 are 
provided in a specific form recorded in a computer read- 
able recording medium, such as flexible disks or CD- 
ROMs. The computer reads the computer programs 

30 from the recording medium and transfers the computer 
programs to either an internal storage device or an ex- 
ternal storage device. Alternatively the computer pro- 
grams may be supplied to the computer via the commu- 
nication path. A microprocessor included in the compu- 

35 ter executes the computer programs stored in the inter- 
nal storage device to attain the functions specified by 
the computer programs. In accordance with an alterna- 
tive procedure, the computer reads the computer pro- 
grams recorded in the recording medium and directly 

40 executes the computer programs. 

[0062] In the specification hereof, the term ■computer 1 
expresses the concept including a hardware device and 
an operating system and thus represents the hardware 
device working under the control of the operating sys- 

45 tern. In the case where the operating system is not re- 
quired but application software alone can activate the 
hardware device, the hardware device itself is equiva- 
lent to the computer. The hardware device includes at 
least a microprocessor like a CPU and means for read- 
so ing the computer programs recorded in the recording 
medium. The computer programs include program 
codes that cause the computer to attain the functions of 
the respective units discussed previously. Part of the 
functions may be attained not by the application soft- 

55 ware but by the operating system. The programs for car- 
rying out the digital watermark embedding process and 
the digital watermark decoding process may be added 
to image processing software in the form of plug-in soft- 


9 


17 


EP1 156 662 A1 


18 


ware programs. 

[0063] Typical examples of the 'recording medium' 
adopted in the present invention include flexible disks, 
CD-ROMs, magneto-optic discs, IC cards, ROM car- 
tridges, punched cards, prints with barcodes or other 
codes printed thereon, internal storage devices (mem- 
ories like a RAM and a ROM) and external storage de- 
vices of the computer, and a variety of other computer 
readable media. 

[0064] The above embodiment is to be considered in 
all aspects as illustrative and not restrictive. There may 
be many modifications, changes, and alterations with- 
out departing from the scope or spirit of the main char- 
acteristics of the present invention. 

Industrial Applicability 

[0065] The technique of the present invention is ap- 
plicable for apparatuses of embedding watermark infor- 
mation in electronic data and apparatuses of electronic 
identification. The technique is effectively applied for 
embedding copyright information and for cryptography 
that includes apparatuses and methods of encryption, 
transmission, and decryption. 


Claims 

1. A method of embedding a digital watermark in a 
master image, said embedding method comprising 
the steps of: 

extracting blocks of a predetermined size from 
said master image; 

processing image data corresponding to each 
block by orthogonal transform; 
comparing orthogonal transformed coefficients 
of at least two blocks having a predetermined 
relationship with each other and making the co- 
efficients satisfy a preset order of magnitude 
according to bit information specified as the 
digital watermark, so as to embed the bit infor- 
mation; and 

processing each block with the embedded bit 
information by inverse orthogonal transform, so 
as to output a resulting image with the digital 
watermark embedded therein. 

2. An embedding method in accordance with claim 1 , 
wherein the predetermined relationship between 
the at least two blocks is an arrangement of conti- 
guity. 

3. An embedding method in accordance with claim 1 , 
wherein the orthogonal transform is discrete cosine 
transform. 

4. An embedding method in accordance with claim 1 , 


said method further comprising the step of: 

quantizing the coefficients obtained by the or- 
thogonal transform with a quantization table 
5 and using the quantized coefficients to embed 

the bit information. 

5. An embedding method in accordance with claim 4, 
said method further comprising the steps of: 

10 

converting the image data into a system of a 
luminance Y and color differences Cb and Cr, 
prior to the orthogonal transform, and carrying 
out discrete cosine transform of the luminance 
15 Y and the color differences Cb and Cr as the 

orthogonal transform; and 
embedding the bit information in coefficients 
obtained by the discrete cosine transform of the 
luminance Y. 

20 

6. An embedding method in accordance with claim 4, 
wherein embedding the bit information is carried out 
when the quantized coefficients of the at least two 
blocks are not all equal to zero. 

25 

7. An embedding method in accordance with claim 1 , 
said method further comprising the steps of: 

introducing a logic function that is true when a 
30 difference between the orthogonal transformed 

coefficients of the at least two blocks having the 
predetermined relationship is in a preset range; 
and 

modifying a procedure adopted to embed the 
35 bit information, based on the true and false 

state of the logic function. 

8. An embedding method in accordance with claim 7, 
said method further comprising the steps of: 

40 

providing a secret key corresponding to each 
coefficient; and 

modifying the procedure adopted to embed the 
bit information, based on the secret key corre- 
45 sponding to each coefficient and the true and 

false state of the logic function with regard to 
the coefficient. 

9. An embedding method in accordance with claim 1 , 
so said method further comprising the steps of: 

providing a basic pattern, which is defined in a 
two-dimensional manner as a combination of 
binary information, as information of the digital 
55 watermark; 

specifying each piece of binary information in- 
cluded in the provided basic pattern as the bit 
information to be embedded; and 
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1 

embedding the binary information of the basic 
pattern by setting the at least two blocks having 
the predetermined relationship to one unit. 

10. An embedding method in accordance with claim 9, 
said method further comprising the step of: 

embedding the basic pattern in the image data 
iteratively a predetermined number of times, 
when the number of elements constituting the 
basic pattern is sufficiently greater than the 
number of extracted blocks. 

11. An embedding method in accordance with claim 9, 
wherein the basic pattern has redundancy 

12. An embedding method in accordance with claim 1 0, 
wherein the basic pattern having redundancy is a 
density pattern. 

13. A method of decoding a digital watermark from a 
master image with the digital watermark embedded 
therein, said decoding method comprising the steps 

of: 

extracting blocks of a predetermined size from 
said master image; 

processing image data corresponding to each 
block by orthogonal transform; and 
comparing orthogonal transformed coefficients 
of at least two blocks having a predetermined 
relationship with each other and extracting bit 
information, based on a preset order of magni- 
tude that is applied to the coefficients. 

14. A decoding method in accordance with claim 13, 
wherein the predetermined relationship between 
the at least two blocks is an arrangement of conti- 
guity. 

15. A decoding method in accordance with claim 13, 
said method further comprising the steps of: 


20 

decoding the digital watermark from the basic 
pattern. 

17. An apparatus of embedding a digital watermark in 
5 a master image, said digital watermark embedding 

apparatus comprising: 

block extraction means that extracts blocks of 
a predetermined size from said master image; 
10 transformation means that processes image 

data corresponding to each block by orthogonal 
transform; 

bit information embedding means that com- 
pares orthogonal transformed coefficients of at 
*5 least two blocks having a predetermined rela- 

tionship with each other and makes the coeffi- 
cients satisfy a preset order of magnitude ac- 
cording to bit information specified as the digital 
watermark, so as to embed the bit information; 
20 and 

output means that processes each block with 
the embedded bit information by inverse or- 
thogonal transform, so as to output a resulting 
image with the digital watermark embedded 
25 therein. 

18. An apparatus of decoding a digital watermark from 
a master image with the digital watermark embed- 
ded therein, said digital watermark decoding appa- 

30 ratus comprising: 

block extraction means that extracts blocks of 
a predetermined size from said master image; 
transformation means that processes image 
35 data corresponding to each block by orthogonal 

transform; and 

bit information extraction means that compares 
orthogonal transformed coefficients of at least 
two blocks having a predetermined relationship 
40 with each other and extracts bit information, 

based on a preset order of magnitude that is 
applied to the coefficients. 
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20 


converting the image data into a system of a 
luminance Y and color differences Cb and Cr, 45 
prior to the orthogonal transform, and carrying 
out discrete cosine transform of the luminance 
Y and the color differences Cb and Cr as the 
orthogonal transform; and 
extracting the bit information, based on coeffi- so 
cients obtained by the discrete cosine trans- 
form of the luminance Y 

16. A decoding method in accordance with claim 13, 
said method further comprising the steps of: 55 

arranging the extracted bit information to re- 
store the basic pattern; and 


19. A recording medium in which a program for embed- 
ding a digital watermark in a master image is record- 
ed in a computer readable manner, said program 
causing a computer to attain the functions of: 

extracting blocks of a predetermined size from 
said master image; 

processing image data corresponding to each 
block by orthogonal transform; 
comparing orthogonal transformed coefficients 
of at least two blocks having a predetermined 
relationship with each other and making the co- 
efficients satisfy a preset order of magnitude 
according to bit information specified as the 
digital watermark, so as to embed the bit infor- 


11 
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mation; and 

processing each block with the embedded bit 
information by inverse orthogonal transform, so 
as to output a resulting image with the digital 
watermark embedded therein. 

20. A recording medium in which a program for decod- 
ing a digital watermark from a master image with 
the digital watermark embedded therein is recorded 
in a computer readable manner, said program caus- 
ing a computer to attain the functions of: 

extracting blocks of a predetermined size from 
said master image; 

processing image data corresponding to each 
block by orthogonal transform; and 
comparing orthogonal transformed coefficients 
of at least two blocks having a predetermined 
relationship with each other and extracting bit 
information, based on a preset order of magni- 
tude that is applied to the coefficients. 
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